Statistical and Algorithmic Approaches to Bullet Matching

Outline

• Same Source Problem: were two bullets fired through the same gun barrel?
• Besides visual inspection: score from Automatic Matching Algorithm based on 3d topographic images
• Score provides objective measure for the match (between 0 and 1, the higher the more likely the match)
• Investigate properties of these scores

x3p format: ISO 25178-72:2017

Automated Matching Algorithm: Front-End Web Application

https://isu-csafe.stat.iastate.edu/shiny/bulletr/

Automated Matching Algorithm

Automated Matching Algorithm: Extract Features

Features are extracted from aligned signaturs \(f\) and \(g\):

• ccf (%) is the maximum value of the Cross-Correlation function, defined as \(C(\tau) = \int_{-\infty}^{\infty} f(t)g(t + \tau)dt\) where \(\tau\) represents the the lag of the second signature (Vorburger et al. 2011).
• D (mm): Euclidean vertical distance between each height value of the aligned signatures, defined as \(D^2 = 1/n \times \sum_t (f(t) - g(t))^2\). (Measure of the total variation, Clarkson and Adams (1933)).
• matches is the number of matching striae between \(f\) and \(g\).

More Features

• mismatches is the number of mismatching striae between \(f\) and \(g\).
• cms is the number of consecutively matching striae between \(f\) and \(g\) (Biasotti 1959).
• non_cms correspondingly the number of consecutively non-matching striae between \(f\) and \(g\).
• S (mm) is the the sum of the average heights of matched striae.

Reference database

• James Hamby Study: Ten consecutively rifled Ruger P-85 pistol barrels used to fire 20 “known” test bullets and 15 “unknown” bullets for comparison (Hamby, Brundage, and Thorpe 2009)
• Use 3d image of Hamby-252 as reference database to train the algorithm
  
• Extract features for all land-land comparisons (21945 total)
• Compare differences in features between 7,075 known matches and 14,040 known non-matches

Distribution of Features

Data-driven rules

• data-driven decision rule (classification tree, Breiman et al 1984)
• tree creates MATCH/NO-MATCH decision for each pair of lands
• in this tree: 1 false positive, 40 false negatives (for pairs of lands)

Algorithm: Forest of 1000 trees

• Algorithm is based on 1000 trees, each tree gets a vote

Feature Importance

• ccf is good, but better in combination with other features

Simulation Study for degradation

1. Three types of Degradation:
   1. Left Fixed - The left portion of the land (leading shoulder) is recoverable.
   2. Middle Fixed - The middle portion of the land is recoverable.
   3. Right Fixed - The right portion of the land (trailing shoulder) is recoverable.
2. Six Degradation Levels: 100% (Fully recovered), 87.5% Recovered, 75% Recovered, 62.5% Recovered, 50% Recovered, 37.5% Recovered, 25% Recovered.

Simulation Results

• Sensitivity: correctly identify a known match (of a pair of lands)
• Specificity: correctly identify a known non-match (of a pair of lands)

Testing this Finding

To come full circle, we will attempt to extract a 50% degraded signature from a Hamby bullet land with bad tank rash in one half. (Barrel 9 Bullet 2 Land 4):

Br924 Results

Extracting the ideal signature and then simulating a left-fixed 50% degradation scenario yields the following:

Testing the Model

Transparency & Reproducibility

• Open Data: NIST ballistics database, locally we are in the process of (re-) scanning Hamby-44 and Hamby-259
• Open Algorithms: bulletr is R package on CRAN with functionality for read/write x3p format, feature extraction and matching algorithms, devel version on github
• Open Results: local database with processed land scans; actual future work: submit scores for land-to-land comparisons from different approaches/institutions/labs to database

Future work: improve matching

• importance of parameter choices: sensitivity analysis, cross-validation
• expand on features, e.g. adapt toolmark scores to bullet lands, such as Chumbley’s U (Chumbley et al. 2010), “diamond score” (PhD thesis by J.Hadler 2017), ccf for different levels of smoothness/resolution
• profile extraction: groove detection, area of extraction, angle of extraction

Future work: other

1. investigate sources of (statistical) error: how much variability is introduced due to operator, lab, type of microscope? (ties into efforts by Martin Baiker from NFI and NIST)
2. expand on applications: e.g. primer shearing marks:
   
3. Aspect of transparency: making data and algorithms accessible to non-expert users

Thank You

Special thanks to Alan Zheng at the National Institute of Standards and Technology for maintaining the NIST Ballistics Toolmark Research Database and providing many useful suggestions.

References